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Abstract Modern commercial Internet search engines display advertisements along 
side the search results in response to user queries. Such sponsored search relies on 
market mechanisms to elicit prices for these advertisements, making use of an auc- 
tion among advertisers who bid in order to have their ads shown for specific key- 
words. We present an overview of the current systems for such auctions and also 
describe the underlying game-theoretic aspects. The game involves three parties — 
advertisers, the search engine, and search users — and we present example research 
directions that emphasize the role of each. The algorithms for bidding and pricing 
in these games use techniques from three mathematical areas: mechanism design, 
optimization, and statistical estimation. Finally, we present some challenges in spon- 
sored search advertising. 



1 Introduction 

Targeted advertisements on search queries is an increasingly important advertising 
medium, attracting large numbers of advertisers and users. When a user poses a 
query, the search engine returns search results together with advertisements that are 
placed into positions, usually arranged linearly down the page, top to bottom. On 
most major search engines, the assignment of ads to positions is determined by an 
auction among all advertisers who placed a bid on a keyword that matches the query. 
The user might click on one or more of the ads, in which case (in the pay-per-click 
modelPay-per-click model) the advertiser receiving the click pays the search engine 
a price determined by the auction. 
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In the past few years, the sponsored search model has been highly successful 
commercially, and the research community is attempting to understand the under- 
lying dynamics, explain the behavior of the market and improve the auction algo- 
rithms. This survey will provide an overview of the algorithmic issues in sponsored 
search. 

The basic view we emphasize is the role of the three partiesThiee parties in 
sponsored search. 

• The first party is the advertisers who have multiple objectives in seeking to place 
advertisements. Some advertisers want to develop their brand, some seek to make 
sales, and yet others advertise for defensive purposes on specific keywords cen- 
tral to their business. Some have budget constraints, while others are willing to 
spend as much as it takes to achieve their goal. Some seek to obtain many clicks 
and eyeballs, yet others attempt to optimize their return on investment. So, in 
general, advertisers are of varied types. 

• The second party is the auctioneer, in this case, the search engine. The search 
engines have to balance many needs. They must maintain useful search results 
and have advertisements enhance, rather than interfere with, the search experi- 
ence. They need to make sure the advertisers get their needs fulfilled, and at the 
same time ensure that the market the advertisers participate in is efficient and 
conducive to business. 

• The third party is perhaps the most important in the game: these are search users. 
Users come to search engines for information and pointers. In addition, they also 
come to discover shopping opportunities, good deals, and new products. There 
are millions of users with different goals and behavior patterns with respect to 
advertisements. 

These three parties induce a fairly sophisticated dynamics. While economic and 
game theory provide a well-developed framework for understanding the auction 
game between the advertisers and the auctioneer, the community has had to gen- 
eralize such methods and apply them carefully to understand the currently popu- 
lar Internet auctions. Likewise, while there has been recent work on understanding 
models of user behavior for posing search queries and their click behavior for search 
responses, little is known about user behavior on advertisements, and crucially, these 
affect the value of the slots and thus the very goods that are sold in auction. 

In this survey, we will show examples of research themes in algorithmic, opti- 
mization and game-theoretic issues in sponsored search. In particular, we present 
three examples each emphasizing the perspective of one of the three different par- 
ties involved in sponsored search: the advertisers (who act as the bidders), the search 
engine (who acts as the auctioneer), and the search engine user (who determines the 
commodity). More specifically, 

• We present results for how an advertiser should choose their bids given the cur- 
rently used auction mechanism and implicit user behavior models. This result 
appears as [ 17 1. It shows that a very simple bidding strategy is very effective for 
the advertiser. 
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• We study a new mechanism for the auctioneer to allocate advertisements to slots 
in order to optimize efficiency, and analyze the game-theoretic aspects of this 
mechanism. This result appears in |fl8l . It shows that a simple price-setting mech- 
anism is suitable for determining the outcome of several auctions simultaneously 
for the auctioneer. 

• We present a novel Markovian model of user behavior when shown advertise- 
ments, and for this model, develop mechanisms and game theory. This result ap- 
pears in (4). It shows that under a model of user behavior more general than the 
one that is implicit in existing auctions, entirely different allocation and pricing 
will be optimal. Hence, user models have significant impact. 

The results above are joint work with Gagan Aggarwal, Evdokia Nikolova, Martin 
Pal and Cliff Stein, and represent work done at Google Research. 

In the rest of the document, we will first describe the foundations behind the 
existing auctions. Then we will describe the three results above. After that, we will 
be able to point to open issues and provide concluding remarks more generally on 
Internet advertising and auctions. 



2 Existing Auctions 

The basic auction behind sponsored searchSponsored search auction occurs when 
a user submits a query to the search engine. The screen shot in Figure [TJ shows an 
example user query "soda" and search results page returned from the search engine. 
This page includes web search results on the left, and independently, a set of three 
text ads on the right, arranged linearly top to bottom, clearly marked "Sponsored 
links." Each advertiser i has previously submitted a bid b, stating their value for a 
click, tying their bid to a specific keyword. The auction is held in real-time among 
advertisers whose keywords match that user's query. The result of the auction is the 
list of advertisements on the right. So, after selecting the set of eligible (matching) 
ads, running the auction involves the search engine determining (a) the ordering of 
bidders and (b) pricing. 

• Ordering: The most natural ordering is to sort by decreasing bid, but that does 
not take into account the quality of ads and their suitability to users. Thus, it 
is common practice to place the bidders in descending order of fc,a,-, where a, 
is what is called the click-through- rate (ctr)Click-through rate (ctr)ctr (click- 
through rate) of advertiser i, i.e., the probability that a user will click on the 
ad, given that the user looks at it. (The ctr is usually measured by the search en- 
gine.) This is the ordering currently in use by search engines like Yahoo! Yahoo! 
and Google. Google 

• Pricing: The natural method is to make bidders pay what they bid, but that leads 
to well-known race conditions 11311 . Instead, the most common method is to use a 
"generalized second price" (GSP) auctionGSP (Generalized Second-Price) auc- 
tion. Say the positions are numbered 1,2, ... starting at the top and going down, 
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Fig. 1 Screen shot of a user query with the search results on the left, and the ads on the right. 



and the bidder at position i has bid b\. In GSP, the price for a click for the ad- 
vertiser in position i is determined by the advertisement below it and is given by 
bi+ 1 oc f + 1 / Of,, which is the minimum they would have needed to bid to attain their 
position. 

The first academic treatments of the sponsored search auction were naturally 
from the perspective of auction and game theory Auction theory. Fixing this ordering 
of the bidders, authors in |[T3l I4T1 [5) focused on understanding the implications of 
different pricing schemes, assuming strategic behavior on the part of the advertisers. 
The setting of the game that is modeled in this work is as follows: each advertiser has 
a private value v* for a click from this user, and wants to set a bid that maximizes her 
utility Uf. The natural economic utility model in this context would be profit:Profit 
utility model i.e., 

Ui = {vi-pi)c u 

where pi is the price per click, and c\ is the probability of a click occurring. Of course 
Ci is determined by the user, and may depend on any number of factors. One com- 
mon model is to assume that the click probability is separable 0: Separable click 
probability if ad i is placed into position j, then c; = a, f}j where a, is the ad-specific 
"click-through rate" and ]3/ is a position-specific visibility factor.Position-specific 
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visibility factor (We will later explore other utility models in Sections[3]and|4]when 
we incorporate budgets, as well as non-separable user models in Section|5]) 

Natural questions in this context include asking whether there is a pure-strategyPure- 
strategy Nash equilibrium Nash equilibriumNash Equilibrium of this game, and an- 
alyzing the economic efficiency and revenue of such equilibria. By economic ef- 
yzc/encyEconomic efficiency we mean the total advertiser value generated by the 
assignment. This is also commonly referred to as the social welfare. Social welfare 
In the context of sponsored search, the efficiency is the sum of the individual ad- 
vertisers' values; i.e., £,-c,-v/, where c,- is the probability that ; will receive a click 
under this assignment and v,- is i's private value for a click. By a pure-strategy Nash 
equilibrium we mean a set of bids such that no single bidder can change her bid and 
increase her utility. 

Among the most desirable properties of a mechanism is to be truthful,Tmthful 
mechanism which is also referred to as being incentive compatible .Incentive com- 
patible This property says that each bidder's best strategy, regardless of the actions 
of other bidders, is simply to report her true value; i.e., submit v; as her bid. Truthful- 
ness immediately implies the existence of a pure-strategy Nash equilibrium (where 
every bidder reports v,-). Furthermore, it is simple to compute economic efficiency, 
since the assignment (and thus the efficiency) is simply a function of the values 
v,-. Unfortunately, it turns out that the GSP auction is not truthful. However, there 
is a pricing scheme that is truthful, which is based on an application of the fa- 
mous Vickrey-Clarke-Groves (VCG) mechanism ||42][T0]|23] VCG (Vickrey-Clarke- 
Groves) mechanism. Furthermore, the GSP auction, while not truthful, still has a 
well-understood pure-strategy Nash equilibrium: 

Theorem 1 (EJ |4TJ gj). Suppose we have a set of bidders participating in a par- 
ticular sponsored search auction. Assume each bidder has a private value and a 
profit-maximizing utility function. Suppose further that the click probabilities are 
separable. Then, the GSP auction is not truthful, but it does have a pure-strategy 
Nash equilibrium whose outcome (in terms of assignment and prices) is equivalent 
to an application of the truthful VCG auction. 

For a more detailed discussion of this line of research, we refer the reader to ||29l . 
Authors in [5 1 also show that under a more general click probability model, there is 
a pricing method that is truthful. (This pricing method reduces to the VCG pricing 
method when the click-through rates are separable.) Furthermore, they show that in 
this more general setting the GSP has a Nash equilibrium that has the same outcome 
as their mechanism. 



2.1 Practical Aspects 

The results described above regard GSP as an isolated auction, abstracting away 
the context of the larger system of which it is a part. While this is useful from a 
modeling perspective, there are many other elements that make sponsored search a 
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more complex environment. Here we list some of those complicating factors, and 
mention examples of work done to address them. 

• Multiple queries, multiple keywords. Each sponsored auction is conducted for 
a particular search engine user with a potentially unique query. There are per- 
haps millions of such queries every day. Advertisers must submit bids on key- 
words, Keywords in sponsored search and cannot adjust those bids on a per-query 
basis. The degree to which the keyword matches a particular query determines 
not only whether the advertiser will participate in the auction (and also who her 
competitors will be), but also can factor into the click-through rate a, that is used 
for ranking. Theorem [T] only applies to the case where the same auction — with 
the same set of advertisers, and the same click-through rates — is repeated, and 
the bids qualify only for that set of auctions. A lot of the work mentioned be- 
low takes on this complication in various ways; we give two such examples in 
Sections [3] and |4] 

• Budgets. In the private-value modelPrivate-value model each advertiser has a 
value v/ per click, but is willing to spend an arbitrary amount to maximize her 
profit. In reality, many advertisers have operating budgets or spending targets, 
and simply want to maximize their value given the constraints of that budget. This 
budget can be reported to the search engine, who can then employ techniques to 
use the budget efficiently. Analysis of incentives becomes more difficult in the 
presence of budgets. This has been addressed in e.g., I8l l33ll32ll34ll32l l2l l35ll39l 
[17] ID, and we discuss two examples in much more detail in Sections [3] and |4] 

• Reserve prices. The major search engines enforce reserve prices, Reserve prices 
in sponsored search dictating the minimum price that an advertiser can pay for 
a click. Sometimes these reserve prices will even be specific to a particular bid- 
der. Reserve prices are useful for controlling quality on the search results page, 
and also have implications for revenue. The effect of reserve prices on the game 
theory of sponsored search is discussed in detail in lfT31 . 

• Interdependent click probabilities. The "separable" assumption implies that an 
advertiser's click probability depends only on the properties and position of her 
own ad. This ignores the other ads on the search results page, which certainly 
affect the user experience, and therefore the click probability of this advertiser. 
We discuss this further in Section|5] 

• Branding. The private click-value model assumes that a click is what the ad- 
vertiser is ultimately interested in. However a branding advertiser could be in- 
terested in her ad appearing in a high position, but not really care whether or 
not it gets a click (other than due to the fact that they only pay if it does). 
(Indeed, a recent empirical study by the Interactive Advertising Bureau and 
Nielsen//NetRatings concluded that higher ad positions in paid search have a 
significant brand awareness effect 11381 .) Thus we might be interested in an auc- 
tion where an advertiser can express the lowest position she is willing to tolerate 
for her ad. This is the approach taken in Q, where Theorem[T]is generalized to 
this setting. 

• Conversions. The private click-value model also assumes that each click is worth 
the same to an advertiser, which is not always the case in practice. Indeed many 
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advertisers track whether or not a click leads to a conversion, Conversions in 
sponsored search which is some sort of event on the linked page (e.g., a sale, 
a sign-up, etc.) Given this data, the advertiser can learn which keywords lead to 
conversions and therefore which clicks are worth more to them. 

• Estimating various parameters. Most work in the context of the game theory 
of sponsored search has assumed that the parameters like click-through rate and 
position visibility are known. However, estimating these parameters is a difficult 
task (e.g., [36 31 1). Indeed, there is an inherent tradeoff between learning these 
parameters and applying them; one cannot learn that an ad has a bad ctr unless it 
is exposed to the user, but then it was a bad idea to show it in the first place. This 
"exploration/exploitation" tradeoff turns out to be related to the "multi-armed 
bandit"Multi-armed bandit problem problem (see e.g. l2Tll44l[T6l '). 

• Incomplete Knowledge. Both the advertisers and the search engine have incom- 
plete knowledge of the "inventory" available to them, since they do not know 
which queries will arrive. In addition the bidders do not know the other bids 
or click-through rates. This makes the advertiser's optimization problem much 
more difficult (see e.g., (28] El 13 [17] |43l [30l |45l |40] E2). From the search en- 
gine's point of view, we can model incomplete knowledge of the future as an 
online algorithm; see e.g. 03 EU |35] IS1 E2I ESI [B EU HU . 



3 The Advertiser's Point of View: Budget Optimization 

The perspective in this section is the advertisers. The challenge from an advertiser's 
point of view is to understand and interact with the auction mechanism. The adver- 
tiser determines a set of keywords of their interest and then must create ads, set the 
bids for each keyword, and provide a total (often daily) budget. 

While the effect of an ad campaign in any medium is a sophisticated phenomenon 
that is difficult to quantify, one commonly accepted (and easily quantified) notion 
in search-based advertising on the Internet is to maximize the number of clicks. The 
Internet search companies are supportive towards advertisers and provide statistics 
about the history of click volumes and prediction about the future performance of 
various keywords. Still, this is a sophisticated problem for the following reasons 
(among others): 

• Individual keywords have significantly different characteristics from each other; 
e.g., while "fishing" is a broad keyword that matches many user queries and 
has many competing advertisers, "humane fishing bait" is a niche keyword that 
matches only a few queries, but might have less competition. 

• There are complex interactions between keywords because a user query may 
match two or more keywords, since the advertiser is trying to cover all the pos- 

1 The choice of keywords is related to the domain-knowledge of the advertiser, user behavior and 
strategic considerations. Internet search companies provide the advertisers with summaries of the 
query traffic which is useful for them to optimize their keyword choices interactively. We do not 
directly address the choice of keywords in this section, which is addressed elsewhere 1391 . 
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sible keywords in some domain. In effect the advertiser ends up competing with 
herself. 

As a result, the advertisers face a challenging optimization problem. The focus of 
the work in ifTTl is to solve this optimization problem. 

Problem Formulation. We present a short discussion and formulation of the opti- 
mization problem faced by advertisers; a more detailed description is in Section [3Tl 

A given advertiser sees the state of the auctions for search-based advertising as 
follows. There is a set K of keywords of interest; in practice, even small advertisers 
typically have a large set K. There is a set Q of queries posed by the users. For each 
query q G Q, there are functions giving the clicks c/ (£>) and cost 9 (fc) that result from 
bidding a particular amount b in the auction for that query, which we will see a more 
formal model of in the next section. There is a bipartite graph G on the two vertex 
sets representing K and Q. For any query q G Q, the neighbors of q in K are the 
keywords that are said to "match" the query g0 

The budget optimization problem is as follows. Given graph G together with the 
functions clicks (/ (-) and cost 9 (-) on the queries, as well as a budget U, determine 
the bids b^ for each keyword k G K such that £ 9 clicks ? (& ? ) is maximized subject to 
^ t/ cost c/ (^) < U, where the "effective bid" b q on a query is some function of the 
keyword bids in the neighborhood of q. 

While we can cast this problem as a traditional optimization problem, there are 
different challenges in practice depending on the advertiser's access to the query 
and graph information, and indeed the reliability of this information (e.g., it could 
be based on unstable historical data). Thus it is important to find solutions to this 
problem that not only get many clicks, but are also simple, robust and less reliant 
on the information. The notion of a "uniform" strategy is defined in Q7'| which is 
essentially a strategy that bids uniformly on all keywords. Since this type of strategy 
obviates the need to know anything about the particulars of the graph, and effectively 
aggregates the click and cost functions on the queries, it is quite robust, and thus 
desirable in practice. What is surprising is that uniform strategy actually performs 
well, which is proved in ifTTl . 

Main Results and Technical Overview. Some positive and negative results are 
given in IfTTl for the budget optimization problem: 

• Nearly all formulations of the problem are NP-Hard. In cases slightly more gen- 
eral than the formulation above, where the clicks have weights, the problem is 
inapproximable better than a factor of 1 — ^, unless P=NP. 

• There is a (1 — 1 /e)-approximation algorithm for the budget optimization prob- 
lem. The strategy found by the algorithm is a two-bid uniform strategy, which 
means that it randomizes between bidding some value b\ on all keywords, and 
bidding some other value b^ on all keywords until the budget is exhausted. 

2 The particulars of the matching rale are determined by the Internet search company; here we treat 
the function as arbitrary. 

3 This type of strategy can also be interpreted as bidding one value (on all keywords) for part of 
the day, and a different value for the rest of the day. 
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This approximation ratio is tight for uniform strategies. There is also a (1/2)- 
approximation algorithm that offers a single-bid uniform strategy, only using one 
value b\. (This is tight for single-bid uniform strategies.) These strategies can be 
computed in time nearly linear in \Q\ + \K\, the input size. 

Uniform strategies may appear to be naive in first consideration because the key- 
words vary significantly in their click and cost functions, and there may be complex 
interaction between them when multiple keywords are relevant to a query. After all, 
the optimum can configure arbitrary bids on each of the keywords. Even for the 
simple case when the graph is a matching, the optimal algorithm involves placing 
different bids on different keywords via a knapsack-like packing (Section l3"Tt . So, it 
might be surprising that a simple two-bid uniform strategy is 63% or more effective 
compared to the optimum. In fact, our proof is stronger, showing that this strategy is 
63% effective against a strictly more powerful adversary who can bid independently 
on the individual queries, i.e., not be constrained by the interaction imposed by the 
graph G. 

We will also look at the simulations conducted in ifTTll using real auction data 
from Google. The results of these simulations suggest that uniform bidding strate- 
gies could be useful in practice. However, important questions remain about (among 
other things) alternate bidding goals, on-line or stochastic bidding models |35l , and 
game-theoretic concerns JHJ, which we briefly discuss in Section [3~4l 



3.1 Modeling a Keyword Auction 

We begin by considering the case of a single keyword that matches a single user 
query. In this section we define the notion of a "query landscape" that describes 
the relationship between the advertiser's bid and what will happen on this query as a 
result of this bid [ 27 1 . This definition will be central to the discussion as we continue 
to more general cases. 

The search results page for a query contains p possible positions in which our ad 
can appear. We denote the highest (most favorable) position by 1 and lowest by p. 
Assuming a separable user model, associated with each position z is a value /3 [z] that 
denotes the click probability if the ad appears in position z0 We assume throughout 
this section that that j3 [i] < j3 [j] if j < i, that is, higher positions receive at least as 
many clicks as lower positions. 

In order to place an ad on this page, we must enter the GSP auction that is carried 
out among all advertisers that have submitted a bid on a keyword that matches the 
user's query. We will refer to such an auction as a query auction,Query auction to 
emphasize that there is an auction for each query rather than for each keyword. 
In GSP, the advertisers are ranked in decreasing order of bid, and each advertiser is 
assigned a price equal to the amount bid by the advertiser below them in the ranking. 



4 We leave out the ad-specific factor a, from this section for clarity, but all the results in |17| 
generalize to this case as well. 
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Let (b [1 ],..., b [p] ) denote the bids of the top p advertisers in this query auction. For 
notational convenience, we assume that b[0] = °° and b[p] = fi{p] = 0. Since the 
auction is a generalized second price auction, higher bids win higher positions; i.e. 
b[i] > b[i + 1]. Suppose that we bid b on some keyword that matches the user's 
query, then our position is defined by the largest b[/] that is at most b, that is, 

pos(b) = argmax(b[/] : b[/] < b). (1) 

i 

Since we only pay if the user clicks (and that happens with probability j3 [/]), our 
expected cost for winning position i would be cost[;] — j3 [i] -b[i], where i = pos(b). 
We use cost^(fo) and clicks^ (b) to denote the expected cost and clicks that result 
from having a bid b that qualifies for a query auction q, and thus 

cost 9 (£) = /3 [i] • b[z] where i = pos(b), (2) 

clicks g (£>) = j3 [i] where ; = pos(b). (3) 

When the context is clear, we drop the subscript q. The following observations 
about cost and clicks follow immediately from the definitions and equations ([T), <J2J 
and (|3). We use R+ to denote the nonnegative reals. 

Proposition 1. For b £ M + , 

1. The tuple fcost (? (/7),clicks c/ (fo)J can only take on one of a finite set of values V q = 
{(cost[l],/J[l]),...,(cost[p],/3 \p})}. 

2. Both cost 9 (Z?) and c\\cksq{b) are non-decreasing functions of b. 

3. Cost-per-click (cpc)cpc ( Cost-Per-Click)Cost-per-click(cpc) cost^^/clicks^Z?) 
is non-decreasing in b, and is always at most the bid; i.e., cost^fr) /clicks^ (b) < 
b. 

Query LandscapesQuery landscape We can summarize the data contained in the 
functions cost(fr) and clicks(fo) as a collection of points in a plot of cost vs. clicks, 
which we refer to as a landscape. Landscape in a query auction For example, for a 
query with four slots, a landscape might look like TableQ] 



bid range 


cost per click 


cost 


clicks 


[$2.60, oo) 


$2.60 


$1.30 


.5 


[$2.00, $2.60) 


$2.00 


$0.90 


.45 


[$1.60, $2.00) 


$1.60 


$0.40 


.25 


[$0.50, $1.60) 


$0.50 


$0.10 


.2 


[$0, $0.50) 


$0 


$0 






Table 1 A landscape for a query 



It is convenient to represent this data graphically as in Figure|2](ignore the dashed 
line for now). Here we graph clicks as a function of cost. Observe that in this graph, 
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the cpc (cost(fo) /clicks (b)) of each point is the reciprocal of the slope of the line 
from the origin to the point. Since cost(fo), clicks(fr) and cost(£>)/clicks(Z?) are non- 
decreasing, the slope of the line from the origin to successive points on the plot 
decreases. This condition is slightly weaker than concavity. 



.5 , 
.4 



$0.50 $1.00 $1.50 

Cost 



Fig. 2 A bid landscape. 



Suppose we would like to solve the budget optimization problem for a single 
query landscape^ As we increase our bid from zero, our cost increases and our 
expected number of clicks increases, and so we simply submit the highest bid such 
that we remain within our budget. 

One problem we see right away is that since there are only a finite set of points 
in this landscape, we may not be able to target arbitrary budgets efficiently. Suppose 
in the example from Table Q] and Figure [2] that we had a budget of $1.00. Bidding 
between $2.00 and $2.60 uses only $0.90, and so we are under-spending. Bidding 
more than $2.60 is not an option, since we would then incur a cost of $1.30 and 
overspend our budget. 

Randomized strategies To rectify this problem and better utilize our available bud- 
get, we allow randomized bidding strategies. Randomized bidding strategy Let 23 
be a distribution on bids b 6 M + . Now we define cost(^) = Eb^lcos^b)] and 
clicks = £t,~,^[clicks(b)]. Graphically, the possible values of (cost(^ , ),clicks(^)) 
lie in the convex hullConvex hull of the landscape points. This is represented in Fig- 
ure E]by the dashed line. 

To find a bid distribution & that maximizes clicks subject to a budget, we sim- 
ply draw a vertical line on the plot where the cost is equal to the budget, and find 
the highest point on this line in the convex hull. This point will always be the con- 
vex combination of at most two original landscape points which themselves lie on 

5 Of course it is a bit unrealistic to imagine that an advertiser would have to worry about a budget 
if only one user query was being considered; however one could imagine multiple instances of the 
same query and the problem scales. 
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the convex hull. Thus, given the point on the convex hull, it is easy to compute a 
distribution on two bids which led to this point. Summarizing, 

Lemma 1. lfT7ll If an advertiser is bidding on one query, subject to a budget U, then 
the optimal strategy is to pick a convex combination of (at most) two bids which are 
at the endpoints of the line on the convex hull at the highest point for cost U. 

There is one subtlety in this formulation. Given any bidding strategy, randomized 
or otherwise, the resulting cost is itself a random variable representing the expected 
cost. Thus if our budget constraint is a hard budget, we have to deal with the dif- 
ficulties that arise if our strategy would be over budget. Therefore, we think of our 
budget constraint as sq/f,Soft budget constraint that is, we only require that our ex- 
pected cost be less than the budget. In practice, the budget is often an average daily 
budget, and thus we don't worry if we exceed it one day, as long as we are meeting 
the budget in expectation. Further, either the advertiser or the search engine (possi- 
bly both), monitor the cost incurred over the day; hence, the advertiser's bid can be 
changed to zero for part of the day, so that the budget is not overspent^ Thus in the 
remainder of this section, we will formulate a budget constraint that only needs to 
be respected in expectation. 

Multiple Queries: a Knapsack Problem As a warm-up, we will consider next the 
case when we have a set of queries, each with its own landscapeKnapsack problem. 
We want to bid on each query independently subject to our budget: the resulting 
optimization problem is a small generalization of the fractional knapsackFmctional 
knapsack problem problem, and was solved in l27ll . 

The first step of the algorithm is to take the convex hull of each landscape, as in 
Figure [2] and remove any landscape points not on the convex hull. Each piecewise 
linear section of the curve represents the incremental number of clicks and cost in- 
curred by moving one's bid from one particular value to another. We regard these 
"pieces" as items in an instance of fractional knapsack with value equal to the incre- 
mental number of clicks and size equal to the incremental cost. More precisely, for 
each piece connecting two consecutive bids b' and b" on the convex hull, we create 
a knapsack item with value [clicks(£>") — clicks(fo')] and size [cost(fc") — cost(fe')]. 
We then emulate the greedy algorithm for knapsack, sorting by value/size (cost-per- 
click), and choosing greedily until the budget is exhausted. 

In this reduction to knapsack we have ignored the fact that some of the pieces 
come from the same landscape and cannot be treated independently. However, since 
each curve is concave, the pieces that come from a particular query curve are in 
increasing order of cost-per-click; thus from each landscape we have chosen for our 
"knapsack" a set of pieces that form a prefix of the curve. 

3.1.1 Keyword Interaction 

Keyword interaction 



See https : // adwords .google . com/ support /bin /answer . py? answer =22183 



for example. 
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In reality, search advertisers can bid on a large set of keywords, each of them 
qualifying for a different (possibly overlapping) set of queries, but most search en- 
gines do not allow an advertiser to appear twice in the same search results pageQ 
Thus, if an advertiser has a bid on two different keywords that match the same query, 
this conflict must be resolved somehow. For example, if an advertiser has a bid out 
on the keywords "shoes" and "high-heel," then if a user issues the query "high-heel 
shoes," it will match on two different keywords. The search engine specifies, in ad- 
vance, a rule for resolution based on the query the keyword and the bid. A natural 
rule is to take the keyword with the highest bid, which we adopt here, but our results 
apply to other resolution rules. 

We model the keyword interaction problem using an undirected bipartite graph 
G = (K\JQ,E) where K is a set of keywords and Q is a set of queries. Each q £ Q has 
an associated landscape, as defined by cost (/ (b) and clicks^ (b). An edge (k,q) £ E 
means that keyword k matches query q. 

\K\ 

The advertiser can control their individual keyword bid vector a £ W + specifying 
a bid ajt for each keyword k£K. (For now, we do not consider randomized bids, but 
we will introduce that shortly.) Given a particular bid vector a on the keywords, we 
use the resolution rule of taking the maximum to define the "effective bid" on query 
q as 



By submitting a bid vector a, the advertiser receives some number of clicks and 
pays some cost on each keyword. We use the term spend to denote the total cost; 
similarly, we use the term traffic to denote the total number of clicks: 



We also allow randomized strategies, where an advertiser gives a distribution 

\K\ 

over bid vectors a £ . The resulting spend and traffic are given by 

spend(,2/) =£ a ^[spend(a)]; traffic^) =£ a ^ [traffic (a)] 
We can now state the problem in its full generality:Budget Optimization Problem 



Budget Optimization 

Input: a budget U, a keyword-query graph G — (KUQ,E), and landscapes 
(costq(-),clicks 9 (-)) for each q £ Q. 

Find: a distribution srf overbid vectors a £ W + such that spend(^) < U and 
traffic( I K/) is maximized. 



7 See |https : / /adwords . google . com/ support /bin /answer .py?answer=1417 9| 
for example. 



b,(a) 



max afc. 

k:(k,q)£E 
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We conclude this section with a small example to illustrate some feature of the 
budget optimization problem. Suppose you have two keywords K = {u,v} and two 
queries Q = {x,y} and edges E = {(u,x),(u,y),(v,y)}. Suppose query x has one 
position with ctr /3 A [ 1 ] = 1.0, and there is one bid b\ = $ 1 . Query y has two positions 
with ctrs J3 V [1] = j8- v [2] = 1.0, and bids b\ = $e and b\ = $1 To get any clicks from 
x, an advertiser must bid at least $1 on u. However, because of the structure of the 
graph, if the advertiser sets b u to $1, then his effective bid is $1 on both x and y. 
Thus he must trade-off between getting the clicks from x and getting the bargain of 
a click for $e that would be possible otherwise. 



3.2 Uniform Bidding Strategies 

As shown in 07], solving the BUDGET OPTIMIZATION problem in its full general- 
ity is difficult. In addition, it may be difficult to reason about strategies that involve 
arbitrary distributions over arbitrary bid vectors. Advertisers generally prefer strate- 
gies that are easy to understand, evaluate and use within their larger goals. With this 
motivation, we look at restricted classes of strategies that we can easily compute, 
explain and analyze. 

We define a uniform bidding strategy to be a distribution srf over bid vectors 
a £ M. + 1 where each bid vector in the distribution is of the form (b,b, . . . ,b) for 
some real-valued bid b. In other words, each vector in the distribution bids the same 
value on every key word. Uniform bidding strategy 

Uniform strategies have several advantages. First, they do not depend on the 
edges of the interaction graph, since all effective bids on queries are the same. Thus, 
they are effective in the face of limited or noisy information about the keyword 
interaction graph. Second, uniform strategies are also independent of the priority 
rule being used. Third, any algorithm that gives an approximation guarantee will 
then be valid for any interaction graph over those keywords and queries. 

Define a two-bid strategy Two-bid strategy to be a uniform strategy which puts 
non-zero weight on at most two bid vectors. Given the landscapes for all the queries, 
we can compute the best uniform strategy in linear time; the proof also directly 
implies that there is always an optimal two-bid strategy: 

Lemma 2. (T7) Given an instance of BUDGET OPTIMIZATION in which there are 
a total of N points in all the landscapes, we can find the best uniform strategy in 
0(NlogN) time. Furthermore, this strategy will always be a two-bid strategy. 

The authors in iflTl also consider single-bid strategies, Single-bid strategy which 
are uniform strategies that put non-zero weight on at most one non-zero vector, i.e. 
advertiser randomizes between bidding a certain amount b* on all keywords, and 
not bidding at all. A single-bid strategy is even easier to implement in practice than 
a two-bid strategy. For example, the search engines often allow advertisers to set a 
maximum daily budget. In this case, the advertiser would simply bid b* until her 
budget runs out, and the ad serving system would remove her from all subsequent 
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auctions until the end of the day. One could also use an "ad scheduling"Ad schedul- 
ing tool offered by some search companiefl to implement this strategy. The best 
single-bid strategy can also be computed easily from the aggregate landscape. The 
optimal strategy for a budget U will either be the point x s.t. cost(x) is as large as 
possible without exceeding U, or a convex combination of zero and the point y, 
where cost(y) is as small as possible while larger than U . 

Approximation Guarantees of Uniform Strategies. In fact, not only are uniform 
strategies easy to optimize over, they are also guaranteed to have good performance 
compared to the optimal solution. In the case of single-bid strategies, we have the 
following: 

Theorem 2. IfTTl There always exists a uniform single-bid strategy that is ^-optimal. 
Furthermore, for any e > there exists an instance for which all single-bid strategies 
are at most (A + e) -optimal. 

For general uniform strategies — where a two-bid strategy is always optimal — 
IfTTl proves a tighter approximation ratio: 

Theorem 3. IfTTl There always exists a uniform bidding strategy that is (1 — i)- 
optimal. Furthermore, for any e > 0, there exists an instance for which all uniform 
strategies are at most (1 — ~ + e) -optimal. 

Thus if given full information about the landscapes, a bidder has an efficient 
strategy to get a large fraction of the available clicks at her budget. But perhaps 
more importantly, these theorems show that the simple uniform bidding heuristic 
can perform well. 



3.3 Experimental Results 

The authors in [17] ran simulations using the data available at GoogleGoogle which 
we briefly summarize here. They took a large advertising campaign, and, using the 
set of keywords in the campaign, computed three different curves (see Figure[3]) for 
three different bidding strategies. The x-axis is the budget (units removed), and the 
y-axis is the number of clicks obtained (again without units) by the optimal bid(s) 
under each respective strategy. "Query bidding" represents the (unachievable) upper 
bound £2, bidding on each query independently. The "uniform bidding" curves rep- 
resent the results of applying the algorithm: "deterministic" uses a single bid level, 
while "randomized" uses a distribution. For reference, we include the lower bound 
of a [e — 1 ) je fraction of the top curve. 

The data clearly demonstrate that the best single uniform bid obtains almost all 
the possible clicks in practice. Of course in a more realistic environment without full 
knowledge, it is not always possible to find the best such bid, so further investigation 

8 See |https : / /adwords . google . com/ support /bin /answer .py?ariswer=3322 7 1 
for example. 
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Fig. 3 An example with real data. 



is required to make this approach useful. However, just knowing that there is such a 
bid available should make the on-line versions of the problem simpler. 



3.4 Extensions 

The algorithmic result presented here gives an intriguing heuristic in practice: bid a 
single value b on all keywords; at the end of the day, if the budget is under-spent, 
adjust b to be higher; if budget is overspent, adjust b to be lower; else, maintain b. 
If the scenario does not change from day to day, this simple strategy will have the 
same theoretical properties as the one-bid strategy, and in practice, is likely to be 
much better. Of course the scenario does change, however, and so coming up with a 
"stochastic" bidding strategyStochastic bidding strategy remains an important open 
direction, explored somewhat by [35 39]. 

Another interesting generalization is to consider weights on the clicks, which is a 
way to model conversions. (A conversion corresponds to an action on the part of the 
user who clicked through to the advertiser site; e.g., a sale or an account sign-up.) 
Finally, we have looked at this system as a black box returning clicks as a function of 
bid, whereas in reality it is a complex repeated game involving multiple advertisers. 
In 1 8 1, it was shown that when a set of advertisers use a strategy similar to the one 
suggested in ifPTl . under a slightly modified first-price auction, the prices approach 
a well-understood market equilibrium. 



Algorithmic Methods for Sponsored Search Advertising 



17 



4 The Search Engine's Point of View: Offline Slot Scheduling 

In the previous section we saw that when we take the GSP auction as given, and 
view the world through the lens of the bidder, the practical problem becomes more 
complex than what the individual auction was designed for. But we could take the 
question back to the search engine and ask if there is a more general mechanism 
that regards the entire day's worth of queries as part of a single overall game. This 
question is addressed in |18j, where the Offline Ad Slot Scheduling problem is de- 
fined: given a set of bidders with bids (per click) and budgets (per day), and a set of 
slots over the entire day where we know the expected number of clicks in each slot, 
find a schedule that places bidders into slots. The schedule must not place a bidder 
into two different slots at the same time. In addition, we must find a price for each 
bidder that does not exceed the bidder's budget constraint, nor their per-click bid. 
(See below for a formal statement of the problem.) 

A good algorithm for this problem will have high revenue. Also, we would like 
the algorithm to be truthful; i.e., each bidder will be incented to report her true bid 
and budget. In order to prove something like this, we need a utility function for the 
bidder that captures the degree to which she is happy with her allocation. Natural 
models in this context (with clicks, bids and budgets) areClick maximization utility 
function click-maximization — where she wishes to maximize her number of clicks 
subject to her personal bid and budget constraints, or profit-maximization — where 
she wishes to maximize her profit (clicks x profit per click). The work in |[T8l is 
focused on click-maximization0 

We present the efficient mechanism of lfl"8l for Offline Ad Slot Scheduling, which 
is truthful under click-maximization. Also, the revenue-optimal mechanism for Of- 
fline Ad Slot Scheduling is not truthful, but has a Nash equilibrium (under the same 
utility model) whose outcome is equivalent to the (fl~8l mechanism; this result is 
strong evidence that the mechanism has desirable revenue properties. 

Problem Definition. The Offline Ad Slot Scheduling problem lfl8l is defined as fol- 
lowsAd slot schedulingOffline ad slot scheduling. We have n > 1 bidders interested 
in clicks. Each bidder ; has a budget fi, and a maximum cost-per-click (max-cpc) 
m,-. Given a number of clicks c,, and a price per click p, the utility m, of bidder i is 
Ci if both the true max-cpc and the true budget are satisfied, and — °° otherwise. In 
other words, m, = c, if p < m, and cip < Be and ut = — °° otherwise. We have n' 
advertising slots where slot ; receives D, clicks during the time interval [0,1]. We 
assume D\ > ■ ■ ■> D n i. 

In a schedule, each bidder is assigned to a set of (slot, time interval) pairs 
(j, [(7,t)), where j < n 1 and < a < T < 1. A feasible sc/iet/t/ZeFeasible sched- 



9 This choice is in part motivated by the presence of budgets, which have a natural interpretation 
in this application: if an overall advertising campaign allocates a fixed portion of its budget to 
online media, then the agent responsible for that budget is incented to spend the entire budget to 
maximize exposure. In contrast, under the profit-maximizing utility, a weak motivation for budgets 
is a limit on liquidity. Also, this choice of utility function is out of analytical necessity: Borgs et 
al. (8) show that under some reasonable assumptions, truthful mechanisms are impossible under a 
profit-maximizing utility. 
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ule in ad slot scheduling is one where no more than one bidder is assigned to a slot 
at any given time, and no bidder is assigned to more than one slot at any given time. 
(In other words, the intervals for a particular slot do not overlap, and the intervals for 
a particular bidder do not overlap.) A feasible schedule can be applied as follows: 
when a user query comes at some time a € [0,1], the schedule for that time instant 
is used to populate the ad slots. If we assume that clicks come at a constant rate 
throughout the interval [0,1], the number of clicks a bidder is expected to receive 
from a schedule is the sum of (t — o)Dj over all pairs {j, [a, t)) in her schedule F°l 

A mechanism for Offline Ad Slot Scheduling takes as input a declared budget 
Bj and declared max-cpc (the "bid") b,, and returns a feasible schedule, as well as 
a price per click p\ < bj for each bidder. The schedule gives some number c, of 
clicks to each bidder i that must respect the budget at the given price; i.e., we have 
pic, < Bj. The revenue of a mechanism is Y,iPi°i- A mechanism is truthful if it is 
a weakly dominant strategy to declare one's true budget and max-cpc; i.e., for any 
bidder i, given any set of bids and budgets declared by the other bidders, declaring 
her true budget B, and max-cpc m; maximizes Uj. In this setting, a (pure strategy) 
Nash equilibrium is a set of declared bids and budgets such that no bidder wants to 
change her declaration of bid or budget, given that all other declarations stay fixed. 
An E-Nash equilibriume-Nash equilibrium is a set of bids and budgets where no 
bidder can increase her u, by more than e by changing her bid or budget. 

Throughout the presentation we assume some arbitrary lexicographic ordering 
on the bidders, that does not necessarily match the subscripts. When we compare 
two bids bi and bf we say that b\ >- b( iff either bi > bf, or bi = bf but i occurs first 
lexicographic ally. 

We comment that for this problem one is tempted to apply a Fisher Market 
model: here m divisible goods are available to n buyers with money Bi, and Ujj(x) 
denotes i's utility of receiving x amount of good j. It is known |6] [14] QT] that 
under certain conditions a vector of prices for goods exists (and can be found effi- 
ciently |[T2l ) such that the market clears, in that there is no surplus of goods, and all 
the money is spent. The natural way to apply a Fisher model to a slot auction is to 
regard the slots as commodities and have the utilities be in proportion to the number 
of clicks. However this becomes problematic because there does not seem to be a 
way to encode the scheduling constraints in the Fisher model; this constraint could 
make an apparently "market-clearing"Market-clearing equilibrium equilibrium in- 
feasible. 



10 All the results of (_18] generalize to the setting where each bidder i has a bidder-specific factor 
a,- in the click-through rate and thus receives (t — o")o£,Dy clicks (see Section |43} . We leave this 
out for clarity. 
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4.1 Special Case: One Slot 

In this section we consider the case k = 1, where there is only one advertising slot, 
with some number D := D\ of clicks. A truthful mechanism for this case is derived 
by first considering the two extreme cases of infinite bids and infinite budgets. 

Suppose all budgets Bi = °°. Then, our input amounts to bids b\ >- bi >- . . . >- b n . 
The obvious mechanism is simply to give all the clicks to the highest bidder. We 
charge bidder 1 her full price p\ =b\. A simple argument shows that reporting 
the truth is a weakly dominant strategy for this mechanism. Clearly all bidders will 
report b[ < m,-, since the price is set to £>, if they win. The losing bidders cannot gain 
from decreasing b\. The winning bidder can lower her price by lowering bi, but this 
will not gain her any more clicks, since she is already getting all D of them. 

Now suppose all bids b, = °°; our input is just a set of budgets B\,. . . ,B n , and 
we need to allocate D clicks, with no ceiling on the per-click price. Here we apply 
a simple rule known as proportional s/ian'ngProportional sharing (see l26l [25 l n f) : 
Let S3 = Now to each bidder i, allocate {B\jSS)D clicks. Set all prices the 
same: p\ = p = S3 /D. The mechanism guarantees that each bidder exactly spends 
her budget, thus no bidder will report > ZJ,. Now suppose some bidder reports 
B\ = Bj — A, for A > 0. Then this bidder is aUocated - A)j(3g - A) clicks, 
which is less than D(Bij SS), since n > 1 and all B, > 0. 

Greedy First-Price Mechanism. A natural mechanism for the general single-slot 
case is to solve the associated "fractional knapsack"Fractional knapsack problem 
problem, and charge bidders their bid; i.e., starting with the highest bidder, greed- 
ily add bidders to the allocation, charging them their bid, until all the clicks are 
allocated. We refer to this as the greedy first-priceGFP (Greedy First-Price) mech- 
anismGreedy first-price (GFP) mechanism (GFP) mechanism. Though natural (and 
revenue-maximizing as a function of bids) this is easily seen to be not truthful: 

Example 1. (T8| Suppose there are two bidders and D = 120 clicks. Bidder 1 has (mi = $2, 
5i = $100) and bidder 2 has (m 2 = $1, B 2 = $50). In the GFP mechanism, if both bidders tell the 
truth, then bidder 1 gets 50 clicks for $2 each, and 50 of the remaining 70 clicks go to bidder 2 for 
$1 each. However, if bidder 1 instead declares bi = $1 + e, then she gets (roughly) 100 clicks, and 
bidder 2 is left with (roughly) 20 clicks. 

The problem here is that the high bidders can get away with bidding lower, thus 
getting a lower price. The difference between this and the unlimited-budget case 
above is that a lower price now results in more clicks. It turns out that in equilibrium, 
this mechanism will result in an allocation where a prefix of the top bidders are 
allocated, but their prices equalize to (roughly) the lowest bid in the prefix (as in the 
example above). 

The Price-Setting Mechanism. An equilibrium allocation of GFP can be computed 
directly via the following mechanism, which |[T8l refers to as the price-setting (PS) 

11 Nguyen and Tardos |37| give a generalization of [251 to general polyhedral constraints, and also 
discuss the application to sponsored search. Both their bidding language and utility function differ 
from 1181 , but it would be interesting to see if there are any connections between their approach 
and d]. 
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mechanism. Price-setting (PS) mechanismPS (Price-Setting) mechanism Essentially 
this is a descending price mechanism: the price stops descending when the bidders 
willing to pay at that price have enough budget to purchase all the clicks. We have 
to be careful at the moment a bidder is added to the pool of the willing bidders; if 
this new bidder has a large enough budget, then suddenly the willing bidders have 
more than enough budget to pay for all of the clicks. To compensate, the mecha- 
nism decreases this "threshold" bidder's effective budget until the clicks are paid 
for exactly. 

Price-Setting (PS) Mechanism (Single Slot) |18| 

• Assume wlog that b\ >- b% >- . . . >- b n > 0. 

• Let k be the first bidder such that b k+i < Y!i=i Bi/D. Compute price p — 
min{Lf =1 B,-/DA.}. 

• Allocate Bj/p clicks to each i <k—l. Allocate B k / p clicks to bidder k, 
where B k = pD - Y*z\ B h 

Example 2. (TH) Suppose there are three bidders with b\ = $2, b 2 = $1, 63 = $0.25 and 
Bj = $100, B 2 = $50, B 3 = $80, and D = 300 clicks. Running the PS mechanism, we get 
k = 2 since Bi/D = 1/3 < b 2 = $1, but (B\ +B 2 )/D = $0.50 > b 3 = $0.25. The price is set 
to min{$0.50,$l} = $0.50, and bidders 1 and 2 get 200 and 100 clicks at that price, respectively. 
There is no threshold bidder. 

Example 3. [181 Suppose now bidder 2 changes her bid to b 2 = $0.40 (everything else remains 
the same as Example|2j. We still get k = 2 since B\ /D = 1/3 < b 2 = $0.40. But now the price is set 
to min{$0.50, $0.40} = $0.40, and bidders 1 and 2 get 250 and 50 clicks at that price, respectively. 
Note that bidder 2 is now a threshold bidder, does not use her entire budget, and gets fewer clicks. 

Theorem 4. Ifl8l The price-setting mechanism (single slot) is truthful. 

Price-Setting Mechanism Computes Nash Equilibrium of GFP. Consider the 
greedy first-price auction in which the highest bidder receives B\/b\ clicks, the 
second ^2/^2 clicks and so on, until the supply of D clicks is exhausted. It is im- 
mediate that truthfully reporting budgets is a dominant strategy in this mechanism, 
since when a bidder is considered, her reported budget is exhausted as much as 
possible, at a fixed price. However, reporting bi = m, is not a dominant strategy. 
Nevertheless, it turns out that GFP has an equilibrium whose outcome is (roughly) 
the same as the PS mechanism. One cannot show that there is a plain Nash equilib- 
rium because of the way ties are resolved lexicographically, so |[T8l proves instead 
that the bidders reach an e-Nash equilibrium: 

Theorem 5. Suppose the PS mechanism is run on the truthful input, resulting in 
price p and clicks ci, . . . ,c„ for each bidder. Then, for any £ > there is a pure- 
strategy £-Nash equilibrium of the GFP mechanism where each bidder receives 
Ci ± £ clicks. 
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4.2 Multiple Slots 

Generalizing to multiple slots makes the scheduling constraint nontrivial. Now in- 
stead of splitting a pool of D clicks arbitrarily, we need to assign clicks that corre- 
spond to a feasible schedule of bidders to slots. The conditions under which this is 
possible add a complexity that needs to be incorporated into the mechanism. 

As in the single-slot case it will be instructive to consider first the cases of infinite 
bids or budgets. Suppose all B, = °°. In this case, the input consists of bids only 
b\ y b% y . . . y b n . Naturally, what we do here is rank by bid, and allocate the slots 
to the bidders in that order. Since each budget is infinite, we can always set the prices 
Pi equal to the bids By the same logic as in the single-slot case, this is easily seen 
to be truthful. In the other case, when bj = °o, there is a lot more work to do. 

Without loss of generality, we may assume the number of slots equals the number 
of bids (i.e., n' = «); if this is not the case, then we add dummy bidders with B, = 
bj = 0, or dummy slots with Z); = 0, as appropriate. We keep this assumption for the 
remainder of the section. 

Assigning Slots Using a Classical Scheduling Algorithm. First we give an im- 
portant lemma that characterizes the conditions under which a set of bidders can 
be allocated to a set of slots, which turns out to be just a restatement of a classical 
result |24] from scheduling theory. 

Lemma 3. l24l [181 Suppose we would like to assign an arbitrary set { 1 , . . . ,k} of 
bidders to a set of slots {l,...,k} with D\ > ■■■ > D^. Then, a click allocation 
c\ > ... > c\ is feasible iff 

cH hc f <£>H YDfi foralU = 1, (4) 

Proof. In scheduling theory, we say a job with service requirement x is a task that 
needs x/s units of time to complete on a machine with speed s. The question of 
whether there is a feasible allocation is equivalent to the following scheduling prob- 
lem: Given k jobs with service requirements xi = c,-, and k machines with speeds 
Sj = Di, is there a schedule of jobs to machines (with preemption allowed) that com- 
pletes in one unit of time? 

As shown in [24, 22], the optimal schedule for this problem (a.k.a. Q\pmtn |C max ) 
can be found efficiently by the level algorithm.Level algorithm and the schedule 
completes in time max^<fc{]£; =1 ^i/Lf=i Thus, the conditions of the lemma are 
exactly the conditions under which the schedule completes in one unit of time. □ 

A Multiple-Slot Budgets-Only Mechanism. The mechanism in lfl8l is roughly a 
descending-price mechanismDescending-price mechanism where we decrease the 
price until a prefix of budgets fits tightly into a prefix of positions at that price, 
whereupon we allocate that prefix, and continue to decrease the price for the re- 
maining bidders. More formally, it can be written as follows: 
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Price-Setting Mechanism (Multiple Slots, Budgets Only) [18| 

• If all Di = 0, assign bidders to slots arbitrarily and exit. 

• Sort the bidders by budget and assume wlog that B\ > B2 > ... >B„. 

• Define ri — £)f =1 fi,/X (=1 A- Set price p = max^ r£. 

• Let £* be the largest £ such that = p. Allocate slots {1 ,■■■£*} to bidders 
{1, ...,£*} at price p, using all of their budgets; i.e., c, = Z?,-/ p. 

• Repeat the steps above on the remaining bidders and slots until all slots are 
allocated. 



Note that the allocation step is always possible since for all £ < £*, we have p > 
r/> = Yli=\Bi/Y?i=\Di, which rewritten is Y!l=\ c i 5= L|=iA'> an d so we can apply 
Lemma[3] An example run of the price-setting mechanism is shown in Figure |4] 



Bidder 


Budget 


■ 1 


$80 


■ 2 


$70 


□ 3 


$20 


□ 4 


$1 
















20/21 



Di = 100 

D 2 = 50 
D 3 =25 
D 4 =0 



Pi =$1.00 



p 2 = $0.84 



Fig. 4 An example of the PS mechanism (multiple slots, budgets only). The first application of 
Find-Price-Block computes n = B1/D1 = 80/100, r 2 = (Si +B 2 )/(Pi +D 2 ) = 150/150, r 3 = 
(81 +B 2 +B 3 )/(D I +D 2 +D 3 ) = 170/175, r 4 = (B, +B 2 +fi 3 +B 4 )/{D 1 +D 2 +D 3 +D 4 ) = 
171/175. Since r 2 is largest, the top two slots make up the first price block with a price p\ — 
r 2 = $1; bidder 1 gets 80 clicks and bidder 2 gets 70 clicks, using the schedule as shown. In the 
second price block, we get B3/D3 = 20/25 and (83 +Bn)/{Dj, +D4) = 21/25. Thus p 2 is set to 
21 /25 = $0.84, bidder 3 gets 500/21 clicks and bidder 4 gets 25/21 clicks, using the schedule as 
shown. 



Theorem 6. |[T8l The price-setting mechanism (multi-slot, budgets only) is truthful. 



The Price-Setting Mechanism (General Case). The generalization of the multiple- 
slot PS mechanism to use both bids and budgets combines the ideas from the bids- 
and-budgets version of the single slot mechanism with the budgets-only version of 
the multiple-slot mechanism. As our price descends, we maintain a set of "active" 
bidders with bids at or above this price, as in the single-slot mechanism. These ac- 
tive bidders are kept ranked by budget, and when the price reaches the point where 
a prefix of bidders fits into a prefix of slots (as in the budgets-only mechanism) we 
allocate them and repeat. As in the single-slot case, we must be careful when a bid- 
der enters the active set and suddenly causes an over-fit; in this case we again reduce 
the budget of this "threshold" bidder until it fits. For details on this mechanism and 
a proof that it is also truthful, we refer the reader to the paper lfl8l . 
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Greedy First-Price Mechanism for Multiple Slots. In the multiple-slot case, as 
in the single-slot case, there is a natural greedy first-price (GFP)Greedy first-price 
(GFP) mechanismGFP (Greedy first-price) mechanism mechanism when the bid- 
ding language includes both bids and budgets: Order the bidders by bid b\ >- b% >- 
... y b n . Starting from the highest bidder, for each bidder i compute the maximum 
possible number of clicks c, that one could allocate to bidder i at price b\, given the 
budget constraint B, and the commitments to previous bidders c\, . . . ,c,-i. This re- 
duces to the "fractional knapsack" problem in the single-slot case, and so one would 
hope that it maximizes revenue for the given bids and budgets, as in the single-slot 
case. This is not immediately clear, but does turn out to be true (see lfl"8l for details). 
As in the single-slot case, the GFP mechanism is not a truthful mechanism. How- 
ever, lfl8l give a generalization of Theorem [5] showing that the multiple-slot GFP 
mechanism does have a pure-strategy equilibrium, and that equilibrium has prices 
and allocation equivalent to the multiple-slot price setting mechanism. 



4.3 Extensions 

There are several natural generalizations of the Online Ad Slot Scheduling problem 
where it would be interesting to extend or apply the results of |[T8l : 

• Click-through rates. To incorporate ad-specific click-through rates a,- into this 
model, we would say that a bidder i assigned to slot j for a time period of length 
T — a would receive (t — <r)a,D ; - clicks. All the results of lfl8l can be gener- 
alized to this setting by simply scaling the bids using b\ = bjttj. However, now 
the mechanism does not necessarily prefer more efficient solutions; i.e., ones that 
generate more overall clicks. It would be interesting to analyze a possible tradeoff 
between efficiency and revenue in this setting. 

• Multiple Keywords. To model multiple keywords in this model, we could say 
that each query q had its own set of click totals D q \ . . .D q>n , and each bidder 
is interested in a subset of queries. The greedy first-price mechanism is easily 
generalized to this case: maximally allocate clicks to bidders in order of their 
bid bt (at price bj) while respecting the budgets, the query preferences, and the 
click commitments to previous bidders. It would not be surprising if there was an 
equilibrium of this extension of the greedy mechanism that could be computed 
directly with a generalization of the PS mechanism. 

• Online queries, uncertain supply. In sponsored search, allocations must be made 
online in response to user queries, and some of the previous literature has focused 
on this aspect of the problem (e.g., I34ll32l ). Perhaps the ideas from |fl8l could 
be used to help make online allocation decisions using (unreliable) estimates of 
the supply, a setting considered in ll32l . with game-theoretic considerations. 
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5 The User's Point of View: a Markov Model for Clicks 

In the GSP auction, by fixing the sort order, we leave out an important third party 
in sponsored search; i.e., the search engine user. Unfortunately, there is very little 
guidance on this in the literature, even though the user's behavior is the essential 
ingredient that defines the commodity the advertisers are bidding on, and its value. 
In 101 a different framework is suggested for principled understanding of sponsored 
search auctions: 

• Define a suitable probabilistic model for search engine user behavior upon being 
presented the ads. 

• Once this model is fixed, ask the traditional mechanism design questions of how 
do assign the ads to slots, and how to price them. 

• Analyze the given mechanism from the perspective of the bidders (e.g., strate- 
gies) and the search engine (e.g., user satisfaction, efficiency and revenue). 

There are certain well-accepted observations about the user's interaction with the 
sponsored search ads that should inform the model: 

• The higher the ad is on the page, the more clicks it gets. 

• The "better" the ad is, the more clicks it gets, where the "goodness" of an ad is 
related to the inherent quality of the ad, and how well it matches the user's query. 

These properties govern not only how the auction is run but also how advertisers 
think about their bidding strategy (they prefer to appear higher and get more clicks). 
Thus it is important for an auction to have what we call intuitive bidding: a higher 
bid translates to a higher position and more clicks. 

In 0, a natural Markov model is proposed for user clicks, taking the above 
observations into account. An algorithm is given to determine an optimal assignment 
of ads to positions in terms of economic efficiency. Together with VCG pricing, this 
gives a truthful auction. They further show that the optimal assignment under this 
model has certain mono tonicity properties that allow for intuitive bidding. In what 
follows, we will describe these contributions in more detail. 

Modeling the Search Engine User. Previous work on sponsored search has (implic- 
itly) modeled the user using two types of parameters: ad-specific click-through rates 
a, and position-specific visibility factors j3,. There are some intuitive user behavior 
models that express overall click-through probabilities in terms of these parameters. 
One possibility is "for each position j independently, the user looks at the ad i in 
that position with probability fij then clicks on the ad with probability a,." Alter- 
natively: "The user picks a single position according to the distribution implied by 
the j3 7 's, and then clicks on the ad i in that position with probability a,." Under both 
these models, it follows that the probability of an ad ; in position j receiving a click 
is equal to a,j3/, which is the so-called separability assumption (see |5 1 or the dis- 
cussion in Section|2]i. From separability it follows that GSP ordering of ads will be 
suitable, because GSP ordering maximizes the total advertiser value on the page. 
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In both these models there is no reason a priori that the position factors fij should 
be decreasing; this is simply imposed because it makes sense, and it is verifiable em- 
pirically. Also, both suggested models assume that the probability of an ad getting 
clicked is independent of other ads that appear with it on the page, an assumption 
made without much justification. It is hard to imagine that seeing an ad, perhaps 
followed by a click, has no effect on the subsequent behavior of the user. 

In designing a user model, we would like to have the monotonicity of the posi- 
tions arise naturally. Also, each ad should have parameters dictating their effect on 
the user both in terms of clicking on that ad, as well as looking at other ads. In J4), a 
model is proposed of a user who starts to scan the list of ads from the top, and makes 
decisions (about whether to click, continue scanning, or give up altogether) based 
on what he sees. More specifically, the user is modeled as the following Markov 
process: "Begin scanning the ads from the top down. When position j is reached, 
click on the ad i with probability a,. Continue scanning with probability g, ." In this 
model, if we try to write the click probability of an ad i in position j as a,j3,, we 
get that [5j = nji eA qji, where A is the set of ads placed abovd 12 ! position j. Thus the 
"position factor" in the click probability decreases with position, and does so natu- 
rally from the model. Also note that we do not have separability anymore, since J3y 
depends on which ads are above position j. Consequently, it can be shown that GSP 
assignment of ads is no longer the most efficient. 

Auction with Markovian users. Given this new user model, we can now ask what 
the best assignment is of ads to slots. In |4|, the most efficient assignment is studied; 
i.e., the one that maximizes total advertiser value derived from user clicks. It turns 
out that the structure of this assignment is different than that of GSP, and indeed 
is more sophisticated than any simple ranking. The presence of the ^,'s requires a 
delicate tradeoff between the click probability of an ad and its effect on the slots 
below it. In (4), certain structural properties of the optimal assignment are identified 
and used to find such an optimal assignment efficiently, not only in polynomial 
time, but in near-linear time. Given this algorithm, a natural candidate for pricing 
is VCG ll42l[T0ll23l . which is clearly truthful in this setting, at least under a profit- 
maximizing utility. 

Intuitive Bidding.Intuitive bidding in sponsored search One of the reasons why 
GSP is successful is perhaps because bidding strategy is intuitive: Under GSP rank- 
ing, if an advertiser bids more, they get to a higher position, and consequently, if 
they bid more, their click probability increases. Now that we have defined a more 
sophisticated assignment function, even though VCG pricing is truthful, the auc- 
tion still may not have these intuitive properties. The main technical result in J4) is 
to show that in the Markov user model, if a mechanism uses the most efficient as- 
signment, indeed position and click probabilities are monotonic in an ad's bid (with 
all other bids fixed), thus preserving this important property. While not surprising, 
position-monotonicity turns out to be rather involved to prove, requiring some del- 



Throughout the section, we will often refer to a position or an ad being "higher" or "above" 
another position or ad; this means that it is earlier on the list, and is looked at first by the user. 
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icate combinatorial arguments, and insights into the optimal substructure of bidder 
assignments. 

In summary, sponsored search auctions are a three party process which can be 
studied by modeling the behavior of users first and then designing suitable mecha- 
nisms to affect the game theory between the advertiser and the search engine. The 
work of J4| sheds some light on the intricate connection between the user models 
and the mechanisms; for example, the sort order of GSP that is currently popular 
(sort by frjOC;) is not optimal under the Markov user model. 



5.1 A Simple Markov User Click Model 

We consider a sponsored search auction with n bidders 88 = {1, . . . ,«} and k posi- 
tions. We will also refer to "ad z," meaning the advertisement submitted by bidder 
i. Each bidder i G 8§ has two parameters, a, and g,. The click-through-rate a, is the 
probability that a user will click on ad ;, given that they look at it. The continuation 
probability q; is the probability that a user will look at the next ad in a list, given 
that they look at ad /Markovian user model. 

Each bidder submits a bid to the auction, representing the amount that they 
value a click. The quantity a,i>, then represents the value of an "impression," i.e., 
how much they value a user looking at their ad. This is commonly referred to as 
their "ecpm.'ujscpm (Expected Cost Per Thousand [Impressions])Expected cost per 
thousand [impressions] (ecpm) Throughout, we will use the notation e,- = (Xjbj for 
convenience. 

Given an assignment (xi,. . . ,JCjfc) of bidders to the k positions, the user looks at 
the first ad x\, clicks on it with probability a xi , and then continues looking with 
probability q Xl This is repeated with the second bidder, etc., until the last ad is 
reached, or some continuation test has failed. Thus the overall expected value of the 
assignment to the bidders is 

e.v, + q.xi {e X2 + q* 2 (e X3 + q X3 (• ■ ■ # v _, («*„))))• 

Now that we have defined the user model, and characterized the value of an 
assignment in that model, we can now define a new auction mechanism: First, 
the search engine computes an assignment of ads to positions that maximizes the 
overall expected value. Given this assignment, prices can then be computed using 
VCG ll42l [TOl |231 : for each assigned bidder we compute the change in others' value 
if that bidder were to disappear. This assures truthful reporting of bids under a profit- 
maximizing utility function. 



The acronym ecpm stands for "expected cost per thousand" impressions, where M is the roman 
numeral for one thousand. We will drop the factor of one thousand and refer to a,£>, as the "ecpm." 
14 The click event and the continuation event could in principle have some correlation, and the 
results mentioned here will still hold. 
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5.2 Properties of Optimal Assignments for Markovian Users 

Since the optimal assignment used by the mechanism is no longer simple ranking 
by ecpm, it is essential to understand the structure of this assignment. This under- 
standing will allow us to compute the assignment more efficiently, and prove some 
important game-theoretic properties of the mechanism. 

It turns out that the quantity e,-/(l — qi), which we will refer to as the "adjusted 
ecpm (a-ecpm)," Adjusted ecpm (a-ecpm)a-ecpm (Adjusted ecpm) plays a central 
role in this model. Intuitively, this quantity is the impression value adjusted by the 
negative effect this ad has on the ads below it. We use a,- = e,/ (1 — qi) for conve- 
nience. The following theorem tells us how to assign a set of k selected ads to the k 
positions: 

Theorem 7. [4] In the most efficient assignment, the ads that are placed are sorted 
in decreasing order of adjusted ecpm a; = e,/ (1 — q,). 

While this theorem tells us how to sort the ads selected, it does not tell us which 
k ads to select. One is tempted to say that choosing the top k ads by a-ecpm would 
do the trick; however the following example proves otherwise: 

Example 4. (4) Suppose we have three bidders and two slots, and the bidders have the following 
parameters: 

Bidder e, q; a/ = e;/(l— qi) 

1 $T~75 4 

2 $2 .2 2.5 

3 $0.85 .8 4.25 

Let's consider some possible assignments and their efficiency. If we use simple ranking by 
ecpm e,, we get the assignment (2, 1), which has efficiency $2 + .2($1) = $2.20. If we use simple 
ranking by a-ecpm «; we get the assignment (3. 1) with efficiency $0.85 + .8($1) = $1.65. It turns 
out that the optimal assignment is ( 1 , 2) with efficiency $ 1 + .75 ($2) = $2.50. The assigned bidders 
are ordered by a-ecpm in the assignment, but are not the top 2 bidders by a-ecpm. 

Now suppose we have the same set of bidders, but now we have three slots. The optimal as- 
signment in this case is (3, 1,2); note how bidder 3 goes from being unassigned to being assigned 
the first position. 

In classical sponsored search with simple ranking, a bidder j can dominate an- 
other bidder ; by having higher ecpm; i.e., bidder j will always appear whenever i 
does, and in a higher position. Example|4]above shows that having a higher ecpm (or 
a-ecpm) does not allow a bidder to dominate another bidder in our new model. How- 
ever, we show that if she has higher ecpm and a-ecpm, then this does suffice. This 
is not only interesting in its own right, it is essential for proving deeper structural 
properties. 

Theorem 8. [4 1 For all bidders ; in an optimal assignment, if some bidder j is not 
in the assignment, and a ; > a, and ej > e,, then we may substitute j for i, and the 
assignment is no worse. 



28 



Jon Feldman and S. Muthukrishnan 



The following theorem shows some subset structure between optimal assign- 
ments to different numbers of slots. This theorem is used to prove position mono- 
tonicity, and is an essential ingredient of the more efficient algorithm for finding the 
optimal assignment. Let OPT(C,y) denote the set of all optimal solutions for filling 
j positions with bidders from the set C. 

Theorem 9. |4| Let j e {l,...,k} be some number of positions, and let C be an 
arbitrary set of bidders. Then, for all S 6 OPT(C, j — 1), there is some S' 6 OPT(C, j) 
where S' D S. 

Finally, we state a main technical theorem of H, which shows that bidding is 
intuitive under a mechanism that maximizes value in the Markovian model. 

Theorem 10. (4) With all other bids fixed, the probability of receiving a click in 
the optimal solution is non-decreasing in one's bid. In addition, the position of a 
particular bidder in the optimal solution is monotonic that bidder's bid. 

This theorem, whose proof relies on all the previous results in this section, im- 
plies that from the perspective of a bidder participating in the auction, all the com- 
plexities of the underlying assignment still do not interfere with the intuitive nature 
of bidding; if you bid more, you still get more clicks, and get to a higher position. 



5.3 Computing the Optimal Assignment 

A simple algorithm for computing the optimal assignment proceeds as follows. First, 
sort the ads in decreasing order of a-ecpm in time (9(«log«). Then, let F(i,j) be 
the efficiency obtained (given that you reach slot j) by filling slots (j, . . . ,k) with 
bidders from the set {/, . . . ,«}. We get the following recurrence: 

F(i, j) = max(F(j + 1 , j + 1 )q t + e i ,F(i+l,j)). 

Solving this recurrence for F( 1 , 1 ) yields the optimal assignment, and can be done in 
0(nk) time. Using the properties about optimal assignments proved in the previous 
section, this can be improved to 

Theorem 11. (4) Consider the auction with n Markovian bidders and k slots. There 
is an optimal assignment which can be determined in 0(«log« + £ 2 log 2 «) time. 



6 Open Issues 

We emphasize three open directions, besides the various game-theoretic and algo- 
rithmic open problems already proposed so far. 
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• Estimating Parameters. In order to run the basic auction and its extensions, the 
search companies need to estimate a number of parameters: ctr, position-specific 
factors, minimum bidder-specific reserve prices, etc. In addition, for operational 
reasons, search engines have to provide traffic estimates to potential advertisers, 
that is, for each keyword, they need to show landscape functions such as the ones 
in Section 3. An open research problem is, given a log of search and ad traffic 
over a significant period of time, design and validate efficient learning methods 
for estimating these parameters, and perhaps, even identify the models that fit the 
variation of these parameters over time. This is a significant research challenge 
since these parameters have intricate dependencies, and in addition, there is a 
long tail effect in the logs, that is, there is a significant amount of rare queries 
and keywords, as well as rare clicks for particular keywords. 

• Grand Simulation. In order for the academic world to develop intuition into the 
world of sponsored search auctions and the associated dynamics, we need a grand 
simulation platform that can generate search traffic, ad inventories, ad clicks, and 
market specifics at the "Internet scale" that search engines face. Such a platform 
will help us understand the many tradeoffs: increasing keywords vs increasing 
budget for a campaign, making better bids vs choosing different search engines, 
choosing to bid for impressions vs clicks vs action, etc. Some auction programs 
are currently available! 15 L but a systematic, large scale effort by academia will 
have tremendous impact for research. 

• Grand Models. In general, we need more detailed models for the behavior of 
users, advertisers as well as the impact of the search engine design on them. We 
described a highly preliminary effort here in which the users were Markovian, 
but more powerful models will also be of great interest. For example, a small 
extension is to make the continuation probability qi a function of location as well, 
which makes the optimization problem more difficult. One can also generalize 
the Markov model to handle arbitrary configurations of ads on a web page (not 
necessarily the linear order in current search results page), or to allow various 
other user states (such as navigating a landing page,Landing page in sponsored 
search that is the page that is the target of an ad). Finally, since page layout can 
be performed dynamically, we could ask what would happen if the layout of a 
web page were a part of the model, which would combine both users as well as 
the search engine into a model. In general, there may be grand, unified models 
that capture the relationship between all the three parties in sponsored search. 



7 Concluding Remarks 

We have discussed algorithmic and game-theoretic issues in auctions for sponsored 
search. 



For example, see |http : / /www, hss . caltech . edu/~ j kg/ j Auctions . html 
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Auctions are used for other products in Internet advertising, for example Google's 
AiiSens^GoogleAdSense (Google) where an Internet publisher (like an online news- 
paper) can sign up with an ad network (in this case Google) to place ads on their 
site. Here, an additional aspect of the problem from the auctioneer's perspective is 
how to target ads, that is, how to choose the keywords from the surrounding context, 
to run auctions like the ones we discussed thus far. This also introduces the fourth 
player in the game, i.e., the publisher, and consequently, the game theory is more 
intricate and largely unexplored. 

Internet ads like sponsored search or AdSense may combine different types of 
ads, i.e., text, image or video ads. Each has its own specifications in terms of di- 
mensions, user engagement and effectiveness. How to combine them into a unified 
auction is an interesting challenge. 

Beyond Internet ads, the Internet medium is also used for enabling ads in tra- 
ditional media including TV, Radio, Print etc. In such cases, the auction problems 
may take on a richer combinatorial component, and also, a component based on abil- 
ity to reserve ad slots ahead of time. The resulting algorithmic and game-theoretic 
problems are largely unexplored in the research community. 
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